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Ocean  Data  Quality  Control 


James  A.  Cummings 


Abstract  Automated  ocean  data  quality  procedures  are  presented  The  procedures 
are  logically  grouped  into  four  stages  of  processing,  which  when  taken  together 
form  a  complete  sensor-to-prediction  quality  control  system.  The  main  features  of 
the  different  ocean  observing  systems  assimilated  by  GODAE  are  presented  along 
with  sources  and  types  of  errors  that  can  occur  in  the  data.  Specific  quality  control 
procedures  are  described  that  test  for  these  errors  as  well  as  more  general  procedures 
that  estimate  the  consistency  of  the  data  across  observing  systems.  Performance 
of  the  external  data  checks  in  the  U.S.  Navy  real-time  ocean  data  quality  system 
is  described.  Finally,  the  importance  of  real-time  ocean  data  quality  control  as  an 
observing  system  monitoring  tool  is  emphasized,  and  some  specific  examples  are 
given  of  new  quality  control  techniques  developed  in  numerical  weather  prediction 
that  have  direct  applicability  in  ocean  data  assimilation  and  forecasting  systems. 


4.1  Introduction 

Observation  data  quality  control  is  a  fundamental  requirement  of  GODAE  ocean 
data  assimilation  systems.  Using  or  accepting  erroneous  data  can  cause  an  invalid 
conclusion  to  be  made  or  an  incorrect  analysis.  Alternatively,  rejecting  extreme, 
but  valid,  data  can  miss  the  detection  of  important  events.  The  goal  of  quality 
control,  therefore,  is  to  reduce  or  eliminate  making  the  wrong  decisions.  Quality 
control  must  correctly  identify  observations  that  are  obviously  in  error,  as  well  as 
the  more  difficult  process  of  identify  ing  measurements  that  fall  within  valid  and 
reasonable  ranges,  but  nevertheless  are  erroneous.  It  is  likely  that  decisions  made 
at  the  quality  control  step  affect  the  success  or  failure  of  the  entire  analysis/forecast 
system. 

Ocean  data  quality  control  is  best  performed  in  stages.  The  first  stage  consists 
of  a  series  of  preliminary  data  sensibility  checks.  Observations  failing  any  one  of 
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these  tests  are  considered  to  have  gross  errors  and  are  removed  from  further  con¬ 
sideration.  The  second  stage  is  based  on  a  complex  quality  control  procedure  where 
observations  are  subjected  to  a  series  of  tests.  Observations  are  not  rejected  after 
immediately  failing  any  one  of  the  quality  control  tests;  rather  the  final  quality' 
control  decision  is  based  on  simultaneous  consideration  of  results  from  all  of  the 
tests.  This  process  uses  a  decision-making  algorithm  where  the  ultimate  fate  of  the 
observation  in  the  analysis/forecast  system  is  decided  (accept,  reject,  schedule  for 
manual  intervention).  The  outcome  of  the  decision-making  algorithm  represents 
the  likelihood  an  observation  contains  a  random  error.  The  third  stage  of  ocean  data 
quality  control  is  performed  by  the  analysis  system  itself.  At  this  point,  the  gross 
and  random  error  characteristics  of  the  observations  have  been  determined,  and 
observations  considered  acceptable  for  the  analysis  have  been  selected.  The  third 
stage  of  the  quality  control  is  designed  to  protect  the  analysis  from  marginally  ac¬ 
ceptable  data  that  have,  for  unknown  reasons,  passed  the  earlier  stages  of  the  quality 
control.  A  final  fourth  stage  of  the  quality  control  is  done  after  the  analysis  and  the 
forecast  have  been  completed  as  part  of  a  system  that  performs  routine  assessment 
of  the  impact  of  assimilating  observations  on  the  reduction  of  model  forecast  error. 
Figure  4.1  illustrates  these  different  stages  of  ocean  data  quality  control  and  the 
flow  of  information  through  the  fully  automated,  real-time  system  operated  by  the 
U.S.  Navy. 

In  this  paper,  various  approaches,  procedures,  and  algorithms  used  to  quality 
control  ocean  observations  are  described.  The  emphasis  is  on  real-time,  fully  au¬ 
tomated  ocean  data  quality  control.  It  is  beyond  the  scope  of  this  paper  to  discuss 
the  wide  variety  of  delayed-mode  or  manual  intervention  quality  control  efforts 
that  have  been  implemented  (e  g.,  Boyer  and  Levitus  1994).  The  paper  is  orga¬ 
nized  as  follows.  Section  2  describes  the  real-time  ocean  observing  systems,  and 
Sect.  3  gives  the  stand  alone,  gross  error  quality  control  procedures  that  are  applied 
to  ocean  observations.  Section  4  provides  descriptions  of  external  quality  control 
data  checks  and  brief  overviews  of  specific  sources  of  error  in  ocean  observing 
systems.  Section  5  describes  how  the  various  independent  external  quality  control 
data  checks  can  be  combined  in  a  quality  control  decision-making  algorithm  and 
gives  some  performance  results  from  the  U.S.  Navy  real-time  ocean  data  quality' 
control  system.  Section  6  outlines  the  internal  consistency  checks  that  are  used  in 
the  assimilation  system  itself,  while  Sect.  7  describes  some  possible  quality  control 
outcomes  from  the  data  impact  system.  Finally,  Sect.  8  provides  a  summary  and 
gives  some  conclusions  on  the  interactions  between  ocean  data  quality  control  and 
observation  monitoring. 


4.2  Ocean  Observing  Systems 

A  wide  variety  of  observation  data  types  are  used  in  GODAE  assimilation  sys¬ 
tems.  The  data  include  both  in  situ  and  remotely  sensed  measurements  from  space. 
As  will  be  discussed,  each  observing  system  has  its  own  unique  data  issues  and 
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Stage  4:  Adjoint 
sensitivities 


Analysis  Components 
(QC  ♦  3DVAR) 


Forecast  Component 


Fig.  4. 1  Chart  showing  flow  of  ocean  observations  through  the  different  stages  of  ocean  data  qual¬ 
ity  control  in  the  U.S.  Navy  global  HYCOM  system.  Stage  I  sensibility  error  checks  are  performed 
on  the  raw  data;  stage  2  external  data  checks  are  performed  in  the  fully  automated  ocean  data 
QC  module;  stage  3  internal  data  checks  are  performed  by  the  iterative  solver  in  the  variational 
assimilation;  and  stage  4  adjoint  sensitivity  calculations  are  done  after  the  forecast  before  the  next 
QC  data  cut  (see  text  for  details).  Note  feedback  of  the  HYCOM  forecast  model  fields  and  predic¬ 
tion  errors  into  the  ocean  data  QC  for  use  as  background  fields  in  the  next  execution  of  the  stage 
2  external  data  error  checks 


quality  control  requirements.  Sources  of  operational  ocean  data  are  described  in 
this  section. 

Most  GODAE  assimilation  centers  receive  in  situ  ocean  observations  over  the 
Global  Telecommunication  System  (GTS).  At  the  current  time,  data  transmitted  via 
the  GTS  are  coded  in  specific  data  type  formats  which  use,  at  most,  two  decimal 
places  for  measurements  of  temperature  and  salinity.  Further,  the  existing  formats 
do  not  allow  for  additional  information  about  the  data  in  the  form  of  quality  flags. 
However,  observational  data  on  the  GTS  are  moving  to  a  new  binary  format  based 
on  BUFR  (Binary’  Universal  Form  for  the  Representation  of  data — a  data  format 
maintained  by  the  World  Meteorological  Organization).  In  this  format,  data  values 
can  be  transmitted  with  more  precision  than  the  existing  text-based  data  formats, 
and  local  tables  can  be  added  to  the  message  that  contain  value  added  or  quality 
assurance  information  from  the  data  provider.  The  move  to  BUFR  on  the  GTS  is  a 
long  process,  scheduled  to  be  completed  for  all  ocean  data  types  in  2016.  In  addition 
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to  the  GTS,  Argo  float  data  are  also  available  at  two  global  data  assembly  centers 
(GDAC):  one  in  the  United  States  at  the  Naval  Research  Laboratory,  Monterey, 
California;  and  the  second  in  France  at  the  Coriolis  Data  Center,  Brest. 

There  is  no  standard  way  satellite  oceanographic  observations  are  distributed 
to  GODAE  assimilation  systems.  In  some  cases,  there  is  a  dedicated  push  from 
the  data  provider  to  the  center.  In  other  cases,  data  are  placed  on  dedicated  servers 
where  the  observations  are  then  pulled  by  the  center.  For  example,  the  GODAE 
High  Resolution  SST  pilot  project  has  been  instrumental  in  setting  up  data  servers 
where  satellite  SST  data  providers  transmit  their  SST  retrievals  in  near-real-time  in 
a  common  format  (Donlon  et  al.  2007).  This  effort  has  made  the  availability  of  SST 
data  from  a  wide  variety  of  satellite  systems  commonplace. 


4.2.1  Ship  and  Buoy  Sea  Surface  Temperature  and  Salinity 

Ship  sea  surface  temperature  (SST)  observation  data  types  are  identified  as  en¬ 
gine  room  intake,  hull  contact  sensor,  or  bucket  temperatures  based  on  type  codes 
contained  in  the  ship  reports  received  over  the  GTS.  Buoy  SST  data  types  are  also 
received  from  the  GTS  and  consist  of  fixed  and  drifting  buoy  reports.  Observ¬ 
ing  systems  that  report  both  in  situ  SST  and  sea  surface  salinity  (SSS)  include 
thermo-salinograph  observations  from  ships  are  sent  over  the  GTS  in  TRAKOB 
reports. 


4.2.2  Satellite  Sea  Surface  Temperature 

There  are  many  sources  of  satellite  SST  observations.  Infrared  satellite  sensors 
include  the  NOAA  and  METOP  Advanced  Very'  High  Resolution  Radiometer 
(AVHRR)  polar  orbiter  4-km  resolution  global  area  coverage  (GAC)  and  1-km 
resolution  local  area  coverage  (LAC)  data.  Note  that  METOP  LAC  retrievals  are 
global,  while  NOAA  LAC  retrievals  are  restricted  to  certain  coastal  areas,  mainly 
in  the  northern  hemisphere.  The  Geostationary  Operational  Environmental  Sat¬ 
ellite  (GOES)  infrared  data  have  a  resolution  of  4-km  and  are  available  from 
the  GOES- 10,  GOES-1 1,  and  GOES- 12  satellites.  The  Meteosat  Second  Genera¬ 
tion  (MSG)  is  also  a  geostationary  infrared  satellite  that  provides  4-km  resolution 
data  centered  over  Europe.  The  Advanced  Microwave  Sensor  Radiometer  Earth 
(AMSR-E)  on  board  the  NASA  Aqua  satellite  provides  global  coverage  of  25- 
km  resolution  microwave  SST.  The  Advanced  Along-Track  Scanning  Radiometer 
(AATSR)  instrument  on  board  the  European  ENV1SAT  satellite  provides  the  first 
routine  measurements  of  a  true  skin  SST  at  1-km  resolution.  Typically,  radiance 
data  in  adjacent  pixels  are  averaged  into  2x2  (NOAA  and  METOP  AVHRR)  or 
3x3  (GOES,  MSG)  bins  before  making  the  SST  retrieval  in  order  to  reduce  sat- 
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ellite  sensor  noise  and  produce  a  more  accurate  SST.  This  process  necessarily 
reduces  the  resolution  of  the  sensor  data  listed  above  to  2-km  LAC,  8-km  GAC, 
and  1 2-km  geostationary  retrievals.  SST  retrievals  from  the  NOAA,  METOP, 
and  GOES  satellites  are  available  from  the  Naval  Oceanographic  Office  (May 
and  Osterman  1998;  May  et  al.  1998).  SST  retrievals  from  AATSR  (Corlett  et  al. 
2006)  and  MSG  (Merchant  et  al.  2008,  2009)  are  available  from  Meteo-France 
and  obtained  from  the  GHRSST  data  server  at  the  Jet  Propulsion  Laboratory,  USA 
(Donlon  et  al.  2007). 


4.2.3  Sea  Ice  Concentration 

The  Special  Sensor  Microwave  Imager  (SSM/1)  and  the  Special  Sensor  Microwave 
Imager/Sounder  (SSM1S)  on  board  the  Defense  Meteorological  Satellite  Program 
(DMSP)  series  of  satellites  provide  routine  observations  of  sea  ice  concentration  at 
approximately  25  km  resolution.  At  the  present  time  there  are  3  SSM/1  (FI  1,  FI 3, 
FI 5)  and  3  SSM1S  (FI 6,  FI 7,  FI 8)  satellites  providing  sea  ice  data. 


4.2.4  Temperature  and  Salinity  Profiles 

Profile  observations  are  reported  from  both  fixed  and  moving  platforms.  Most  pro¬ 
file  observations  report  only  temperature,  such  as  expendable  bathythermographs 
(XBT)  and  some  fixed  buoys,  but  profiling  floats  (Argo),  conductivity-tempera- 
ture-depth  (CTD)  sensors,  and  an  increasing  number  of  fixed  buoys  report  both 
temperature  and  salinity.  Profile  observations  are  also  available  from  gliders  that 
measure  temperature  and  salinity  at  varying  depths  and  positions  along  a  dive.  A 
glider  dive  consists  of  a  descending  and  an  ascending  profile  in  which  the  latitude, 
longitude  positions  and  times  of  the  observations  change  with  depth.  The  presence 
oftemperature  and  salinity  in  some  reports  and  the  unique  sampling  characteristics 
of  ocean  gliders  present  new  challenges  to  the  quality  control  of  ocean  profile  data. 
There  are  common  approaches  to  the  quality  control  of  the  various  profile  observ¬ 
ing  systems,  but  there  are  also  unique  instrumentation  specific  error  checks  that 
are  performed. 


4.2.5  Altimeter  Sea  Surface  Height 

At  the  present  time,  sea  surface  height  anomaly  (SSHA)  observations  are  available 
from  satellite  radar  altimeters  on  board  the  Jason- 1,  Jason-2,  and  ENVISAT  satel¬ 
lites.  Historically,  satellite  altimeters  have  been  flown  on  Topex/Poseidon,  Geo- 
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sat  and  Geosat  Follow-on,  and  the  European  Remote  Sensing  series  of  satellites 
(ERS-1  and  ERS-2). 


4.2.6  Altimeter  and  Buoy  Significant  Wave  Height 

Each  satellite  radar  altimeter  also  provides  significant  wave  height  (SWH)  and  wind 
speed  observations.  SWH  observations  are  also  available  from  many  fixed  buoy 
locations,  mainly  in  the  northern  hemisphere.  SWH  observations  are  assimilated 
into  wave  models. 


4.3  Preliminary  Data  Sensibility  Checks 

Several  preliminary  data  sensibility  error  checks  are  performed  prior  to  the  quality 
control  of  the  observed  values.  Observations  failing  any  one  of  these  preliminary 
data  checks  are  considered  to  have  gross  errors  and  are  discarded  or  flagged  for 
rejection.  In  some  cases  the  preliminary  data  checks  are  performed  by  the  data  pro¬ 
vider  and  the  observations  are  simply  not  distributed.  The  preliminary  data  checks 
and  logic  for  accepting/rejecting  observations  at  this  stage  of  the  quality  control 
process  are  described  below. 


4.3.1  Land/Sea/Fresh  Water  Test 

All  observation  locations  are  checked  against  a  global,  high-resolution  land/sea  da¬ 
tabase.  Observation  locations  surrounded  by  water  in  all  directions  are  accepted, 
and  observation  locations  surrounded  by  land  in  all  directions  are  discarded.  For 
observations  very  near  the  coast  a  fuzzy  land/sea  boundary  check  is  used.  Observa¬ 
tion  locations  are  accepted  if,  in  any  one  direction,  the  otherwise  over-land  loca¬ 
tion  is  next  to  a  water  point.  Relaxation  of  the  land/sea  discrimination  allows  for 
resolution  errors  of  the  land/sea  database  and  precision  errors  in  the  reporting  of 
observation  latitude  and  longitude  positions  over  the  GTS.  Fuzzy  land/sea  tests  are 
useful  when  ships  provide  observations  while  parked  at  a  dock,  or  when  instru¬ 
ments  are  deployed  on  piers  very  close  to  the  coast.  Note  that  the  land/sea  test  must 
also  distinguish  observation  locations  in  fresh-  and  seawater  locations.  Lake  surface 
temperatures  are  routinely  provided  in  remotely  sensed  satellite  data  streams,  and 
in  situ  fixed  buoys  are  located  in  many  large  lake  systems,  such  as  the  Great  Lakes 
in  the  U.S.  Remotely  sensed  and  in  situ  lake  surface  temperatures  have  unique  error 
characteristics  and  need  to  be  distinguished  as  such  in  the  quality  control.  Remotely 
sensed  lake  surface  temperatures  are  routinely  used  in  analyses  of  lower  boundary 
conditions  in  Numerical  Weather  Predictions  (NWP)  systems. 
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4,3.2  Location/Speed  Test 

A  location  (speed)  test  is  used  to  determine  if  the  reported  position  of  an  observa¬ 
tion  is  consistent  with  prior  positions  from  the  same  platform.  The  test  is  neces¬ 
sarily  restricted  to  data  types  that  report  unique  call  signs,  such  as  Argo  floats,  sur¬ 
face  ships,  aircraft,  and  fixed  and  drifting  buoys.  Observations  failing  the  location/ 
speed  test  are  typically  scheduled  for  manual  intervention.  No  automated  method 
exists  at  the  present  time  to  correct  erroneous  reported  positions.  The  speed  test 
uses  a  sliding  time  window  of  the  last  25  reported  platform  positions  from  a  given 
platform.  The  algorithm  is  applied  to  sequential  locations  both  forward  and  back¬ 
ward  in  time.  Newly  received  observation  locations  may  appear  to  be  erroneous 
but  in  fact  are  correct  and  indicate  an  error  in  a  past  reported  position.  Forward 
and  backward  application  of  the  speed  test  has  been  shown  to  be  the  best  way  to 
detect  position  errors  in  the  past.  The  method  takes  into  account  differences  in  the 
expected  movement  of  airborne  versus  surface  ship  platforms,  as  well  as  a  test  to 
ensure  that  the  identification  of  a  buoy  is  fixed  or  drifting  based  on  the  expected 
pattern  of  all  numeric  buoy  call  signs.  Care  is  taken  to  ensure  that  the  speed  test 
does  not  inadvertently  reject  new  drifting  buoy  observations  when  drifting  buoy 
call  signs  are  reused.  The  practice  of  reusing  buoy  call  signs  often  results  in  very 
large  changes  from  the  last  reported  position  of  the  failed  buoy  and  the  position 
of  the  new  buoy  with  the  same  call  sign.  This  problem  is  minimized  by  not  check¬ 
ing  locations  from  buoys  with  the  same  call  sign  that  have  reporting  times  which 
differ  by  more  than  120  hours.  Observations  are  rejected  if  the  reported  position 
differs  by  tw  ice  the  expected  rate  of  speed  as  computed  from  the  recent  time  his¬ 
tory  of  platform  locations.  Buoy  drift  direction  is  not  taken  into  account  in  the 
speed  test. 


4.3.3  Valid  Value  Range  Tests 

Valid  value  range  tests  are  applied  to  observed  variables  as  well  as  observation  loca¬ 
tions  and  sampling  times.  Reported  values  of  temperature  are  required  to  be  great¬ 
er  than  -2.5°C  and  less  than  42°C.  Reported  values  of  salinity  are  required  to  be 
greater  than  or  equal  to  0  PSIJ  and  less  than  42  PSU.  Geographic  dependencies  can 
be  built  into  the  temperature  and  salinity  valid  value  range  test  to  take  into  account 
unique  oceanographic  conditions  in  marginal  seas,  such  as  the  Red  Sea,  Mediter¬ 
ranean  Sea,  Sulu  Sea,  and  Black  Sea.  Observation  latitudes  must  be  between  -90 
and  90°,  and  observation  longitudes  must  be  between  -180  and  1 80°.  Current  speed 
observations  are  required  to  be  positive  numbers  and  less  than  a  maximum  value 
of  2  ms-1.  Observation  sampling  times  must  contain  valid  year,  month,  day,  hour, 
minute,  and  second  date-time  information,  and  the  combined  date  time  group  can¬ 
not  formally  lie  in  the  future  (observation  time  younger  than  the  receipt  time  of  the 
observation  at  the  center). 
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4.3.4  Duplicates 

Detection  of  duplicate  reports  in  ocean  observational  data  is  a  difficult  and  recur¬ 
ring  problem.  The  same  data  message  can  be  received  multiple  times  over  different 
networks  at  the  operational  centers.  One  powerful  method  of  detecting  duplicates 
is  to  use  the  cycle  redundancy  check  (CRC).  The  CRC  checksum  is  calculated  as 
a  function  of  the  message  contents.  It  processes  each  byte  of  a  message  file.  Any 
change,  no  matter  how  small,  will  produce  a  different  CRC  number.  Identical  CRC 
numbers,  therefore,  will  indicate  an  exact  duplicate  message  and  one  of  the  mes¬ 
sages  can  simply  be  deleted.  The  real  problem  with  detecting  duplicates,  however, 
is  the  issue  of  near-duplicates;  observations  with  the  same  location,  time  and  exact 
match  of  data  type,  but  otherwise  are  different.  It  may  be  that  one  version  has  lower 
precision,  fewer  reporting  levels,  or  a  different  reporting  source.  It  is  not  possible  to 
reject  near-duplicates  at  the  preliminary  data  check  level  of  quality  control  process¬ 
ing.  Further,  examination  of  the  observed  data  values  is  usually  needed  to  make  an 
informed  decision.  Near-duplicates,  therefore,  are  typically  processed  through  the 
external  data  checks  described  in  Sect.  4  as  unique  data  reports.  It  is  when  updating 
observations  in  the  quality  controlled  data  base  that  the  problem  of  near-duplicates 
needs  to  be  resolved.  It  is  not  advisable  to  maintain  near-duplicates  in  the  data  base 
because  of  the  possibility  of  data  inconsistency.  Accordingly,  a  decision  has  to  be 
made  on  which  of  the  near-duplicate  observations  to  keep  and  which  near-duplicate 
to  toss.  The  decision  should  be  based  on  objective  measures  of  data  quality;  retain 
the  observation  that  has  more  reporting  levels,  has  sampled  deeper,  or  has  received 
better  quality  control  scores. 

An  additional  determination  of  near-duplicates  is  done  when  observations  are 
read  into  the  analysis.  In  this  case,  multiple  observations  can  be  closely  spaced 
(within  a  model  grid  cell)  and  must  be  thinned  horizontally  to  ensure  that  the  co- 
variance  matrix  is  not  ill  conditioned.  Decisions  to  retain  or  toss  observations  at 
this  point  in  the  assimilation  can  take  into  account  additional  information  about  the 
observation,  such  as  data  type  and  quality  control  outcomes  from  the  external  data 
checks  described  in  the  next  section. 


4.4  External  Data  Checks 

Effective  quality  control  is  a  strong  function  of  the  amount  of  information  available. 
A  primary  purpose  of  the  quality  control  system  is  to  gather  validated  information 
about  a  newly  received  observation  in  order  to  determine  the  consistency  of  the 
reported  values  with  what  is  known  about  the  observed  variable.  Knowledge  of  the 
uncertainty  of  the  observation  and  the  collocated  information  is  also  needed  to  for¬ 
mulate  and  test  hypotheses  in  the  quality  control  decision-making  process.  This  in¬ 
formation  is  acquired  and  combined  in  a  series  of  external  data  checks  that  are  per¬ 
formed  prior  to  the  analysis.  Many  of  these  pre-analysis  quality  control  procedures 
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are  specific  to  an  observing  system  and  test  for  instrument  failure  or  known  biases 
in  certain  data  types.  Other  pre-analysis  quality  control  procedures  are  common  to 
more  than  one  data  type.  In  this  category,  background  field  checks  and  cross  vali¬ 
dation  analyses  are  particularly  important.  Pre-analysis  procedures  in  common  are 
described  first  (Sects.  4.1  and  4.2),  followed  by  descriptions  of  procedures  unique 
to  specific  data  types  (Sects.  4.3  through  4.8). 


4J.1  Background  Field  Check 

Background  fields  used  to  quality  control  ocean  data  include  climatology,  short¬ 
term  forecasts,  and  global  or  regional  analyses.  In  all  cases,  appropriate  background 
error  variances  must  be  used.  Background  and  background  error  fields  valid  at  the 
observation  sampling  time  are  interpolated  to  the  observation  location.  An  innova¬ 
tion  is  formed  (observation  minus  background)  and  normalized  by  the  error  esti¬ 
mate  of  the  background  field.  Assuming  errors  are  normally  distributed,  the  prob¬ 
ability  the  observation  contains  a  random  error  is  computed  by, 


(4.1) 


where  x  is  the  observed  value,  p  is  the  background  value,  o  is  the  background  er¬ 
ror  standard  deviation,  and  p  is  the  area  to  the  left  of  X  beneath  the  standardized 
normal  probability  curve.  Histograms  should  be  examined  and  formal  statistical 
tests  performed  to  show  that  the  normalized  background  innovations  are  indeed 
normally  distributed  in  order  to  use  the  probability  of  error  values  to  accept  or  reject 
observations  in  the  quality  control  decision  making  algorithm  (see  Sect.  5).  As  an 
example,  Fig.  4.2  shows  frequency  distributions  of  global  and  regional  analyses 
and  climate  innovations  for  a  6-hour  data  cut  of  SST  retrievals  from  two  different 
satellites  (AMSR-E  and  METOP-A).  The  shapes  of  the  innovation  histograms  for 
the  analysis  backgrounds  clearly  resemble  normal  distributions,  albeit  with  differ¬ 
ent  variances.  The  climate  background  histograms,  however,  are  skewed  with  a  long 
positive  tail.  This  feature  is  most  notable  in  the  METOP-A  data,  and  likely  indicates 
SST  retrievals  from  diurnal  warming  events  that  are  not  represented  in  the  climate 
fields. 


4.4.2  Cross  Validation 

Cross  validation  compares  observations  against  other  nearby  data.  A  variety  of  meth¬ 
ods  are  used  to  make  these  comparisons.  The  most  common  approach  is  to  perform 
an  optimum  interpolation  (01)  analysis  at  the  observation  location  and  sampling  time 
using  nearby  validated  data,  excluding  the  datum  being  checked.  The  innovations 
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Fig.  4.2  Geographic  coverage  charts  and  histograms  of  AMSR-E  and  METOP  LAC  retrieved  SST 
minus  global  (red)  and  regional  (green)  analysis  and  climate  (blue)  backgrounds.  The  AMSR-E 
data  cut  processed  1,369,870  observations  on  28  Dec  2009  at  187.  The  METOP  LAC  data  cut 
processed  2,281,094  observations  on  10  SFP  2009  at  01Z.  Daytime  retrievals  are  indicated  as 
blue  and  nighttime  retrievals  are  indicated  as  green  points  in  the  geographic  coverage  charts.  The 
histograms  are  formed  using  0  25°C  temperature  difference  bins 


for  the  cross  validation  are  computed  from  an  ocean  climatology.  It  is  important  to 
ensure  that  cross  validation  checks  are  data  driven  and  independent  of  any  analysis 
or  forecast  model  backgrounds.  The  uncertainty  of  the  analyzed  value  is  computed 
from  the  OI  analysis  error  reduction  of  climate  variability.  The  cross  validation  ana¬ 
lyzed  value  and  its  uncertainty  are  then  used  as  the  background  and  background  error 
values  in  the  background  field  check  described  in  Sect.  4.1.  In  the  absence  of  any 
nearby  valid  data,  the  cross  validation  procedure  simply  returns  climate  and  climate 
variability  as  the  analysis  and  error  estimates,  and  the  cross  validation  check  is  iden¬ 
tical  to  the  background  check  using  climatology.  Thus,  cross  validation  is  analogous 
to  checking  observations  against  a  dynamic,  time-dependent  climatology. 

The  background  error  covariances  used  in  the  cross  validation  procedure  can 
be  very  simple,  such  as  only  including  data  within  some  specified  distance  from 
the  observation  being  checked,  or  more  complicated,  based  on  the  multivariate 
covariances  used  in  the  assimilation  procedure  itself.  Cross  validation  can  be  ap¬ 
plied  to  all  observation  data  pairs  in  the  quality  control  or  it  can  be  preceded  by 
other  data  checks  which  first  detect  suspect  observations.  The  cross  validation  is 
then  performed  only  on  the  suspect  observations  to  save  on  computational  time. 
In  data  sparse  areas  the  cross  validation  check  will  have  limited  effect.  However, 
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the  continuing  development  of  the  Argo  profiling  float  array  generally  provides  an 
adequate  number  of  nearby  data  to  allow  the  cross  validation  of  profile  observa¬ 
tions  to  work  well  in  practice.  Cross  validation  is  also  useful  in  the  quality  control 
of  altimeter  SSH  and  SWH  observations,  since  individually  those  data  tend  to  be 
rejected  along  sequential  segments  of  altimeter  tracks  due  to  phase  errors  in  the 
model  background  fields. 


4.4.3  Ship  and  Buoy  Sea  Surface  Temperature 

Volunteer  observing  ship  (VOS)  temperatures  have  very  different  error  character¬ 
istics  depending  on  the  measurement  method.  Hull  contact  sensor  measurements 
of  temperature  appear  to  be  the  most  accurate  followed  by  engine  room  intake  and 
buckets.  However,  all  ship-based  measurement  systems  are  prone  to  error  since  the 
on-board  instruments  are  rarely  calibrated.  In  general,  ship-based  SST  measure¬ 
ments  are  noisy  and  observations  from  engine-room-intake  instruments  tend  to  be 
warm  biased,  while  bucket  measurements  are  biased  toward  cooler  temperatures.  In 
addition,  there  appears  to  be  some  geographic  dependence  in  ship  SST  errors,  with 
errors  higher  in  the  Pacific  than  in  the  Atlantic. 

Drifting  buoy  measurements  of  SST  are  very  important  since  the  buoys  are  glob¬ 
ally  distributed  and  have  a  relatively  long  life.  In  general,  drifting  buoy  SST  mea¬ 
surements  are  of  high  accuracy  and  high  quality.  Occasionally,  spurious  drifter  loca¬ 
tions  are  received,  but  these  are  usually  detected  in  the  location  speed  check.  In  gen¬ 
eral,  buoy  SST  measurements  are  quality  controlled  by  the  background  field  checks 
using  climatology  or  analysis  fields.  Since  drifting  buoys  and  ships  are  identified 
by  unique  call  signs  the  time  history  of  individual  instruments  can  be  monitored 
for  indications  of  drift  and  calibration  errors.  Drifters  are  deployed  with  holey-sock 
drogues  to  a  depth  of  —  1 5  m.  Monitoring  surface  drifters  for  drogue  loss  is  impor¬ 
tant,  because  loss  of  a  drogue  changes  the  sampling  characteristics  of  the  drifter 


4.4.4  Satellite  Sea  Surface  Temperature 

Infrared  and  microwave  satellite  SST  retrievals  measure  very  different  properties 
of  the  sea  surface,  requiring  unique  quality  control  procedures.  In  the  sections  to 
follow,  residual  cloud  and  aerosol  contamination  quality  control  tests  are  applied 
to  infrared  SST  retrievals.  Diurnal  warming  detection  is  performed  for  daytime  re¬ 
trievals  from  both  infrared  and  microwave  satellites. 


4.4. 4.1  Residual  Cloud  Contamination 

Infrared  SST  measurements  are  derived  from  radiometric  observations  at  wave¬ 
lengths  of  ~3.7  pm  and  1 1  — 12  pm.  Though  the  3.7  pm  channel  is  more  sensitive  to 
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SST,  it  is  primarily  used  only  for  night-time  measurements  because  of  the  relatively 
strong  reflection  of  solar  irradiation  in  this  wavelength  region  during  daytime,  which 
contaminates  the  retrieved  radiation.  The  infrared  wavelength  bands  are  sensitive  to 
the  presence  of  clouds  and  atmospheric  water  vapor.  For  this  reason,  thermal  infra¬ 
red  measurements  of  SST  first  require  atmospheric  correction  of  the  retrieved  signal 
and  can  only  be  made  for  cloud-free  pixels.  However,  cloud  clearing  algorithms  are 
far  from  perfect  and  atmospheric  water  vapor  variations  are  significant.  In  addition, 
satellite  zenith  angle  plays  a  role  in  determining  SST  errors,  since  the  atmospheric 
path  length  over  which  the  radiation  is  observed  is  longer  at  higher  zenith  angles. 
Residual  cloud  contamination  errors  are  manifested  as  cold  biases.  Detection  of 
these  errors  is  performed  by  the  background  field  check  (Sect.  4.1). 


4.4.4. 2  Aerosol  Contamination 

Satellite  sea  surface  temperature  retrievals  from  infrared  radiometers  are  known  to 
be  prone  to  bias  when  significant  amounts  of  aerosol  are  present  in  the  atmosphere. 
Retrievals  are  degraded  by  the  presence  of  tropospheric  aerosols,  as  are  cloud  de¬ 
tection  tests  that  depend  upon  accurate  visible  and  infrared  channel  measurements. 
In  particular,  desert  dust  particles  are  large  enough  to  attenuate  and  contribute  to 
the  infrared  signal  emitted  from  the  ocean  surface  before  it  reaches  the  satellite 
sensor  in  space.  Saharan  dust  events  are  common  in  the  eastern  tropical  Atlantic 
and  Mediterranean  Sea.  Saharan  dust  is  lifted  by  convection  over  hot  desert  areas, 
and  can  reach  very  high  altitudes;  from  there  it  can  be  transported  over  the  ocean 
by  winds,  covering  distances  of  thousands  of  kilometers.  The  dust  combined  with 
the  hot  dry  air  of  the  Sahara  Desert  has  significant  effects  on  tropical  weather,  es¬ 
pecially  as  it  interferes  with  the  development  of  hurricanes.  In  Eastern  Asia,  min¬ 
eral  dust  events  originate  in  springtime  in  the  Gobi  Desert  (Southern  Mongolia  and 
Northern  China).  The  aerosols  are  carried  eastward  by  prevailing  winds,  and  pass 
over  China,  Korea,  and  Japan,  sometimes  as  far  as  the  western  United  States.  Thus, 
the  impact  of  atmospheric  aerosols  on  infrared  SST  retrievals  is  a  cold  bias  that  is 
a  global  problem. 

The  current  limiting  factor  for  dealing  with  aerosol  contamination  in  satellite 
SST  retrievals  is  accurate  knowledge  of  the  characteristics  and  amount  of  the  aero¬ 
sol  at  the  coincident  time  and  location  of  the  satellite  SST  retrievals.  This  informa¬ 
tion  is  available  in  the  daytime  for  the  anti-solar  side  of  the  scan  in  the  visible  chan¬ 
nels  of  the  instrument  (-25%  of  the  data),  but  there  is  no  information  at  night  or  on 
the  solar  side  of  scan  due  to  the  effects  of  sun  glint  (-75%  of  the  data).  However, 
aerosol  transport  models  can  be  used  to  provide  the  necessary  information.  In  par¬ 
ticular,  the  Navy  Aerosol  Analysis  Prediction  System  (NAAPS)  provides  3-hourly, 
aerosol  optical  depth  (AOD)  forecasts  for  four  aerosol  sources  (dust,  smoke,  sul¬ 
phate,  and  sea  spray)  at  19  different  wavelengths;  14  of  the  NAAPS  aerosol  optical 
depth  wavelengths  match  the  channels  used  in  satellite  SST  retrieval  algorithms. 

The  wavelength  dependent,  global  NAAPS  optical  depth  products  are  used  in  a 
canonical  variate  analysis  to  detect  aerosol  contamination.  Canonical  variate  analy- 
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sis  finds  the  linear  combination  of  observed  variables  that  maximize  the  ratio  of 
between-group  to  within-group  variation.  There  are  five  groups  of  infrared  satellite 
SST  retrievals:  four  groups  are  defined  with  varying  levels  of  aerosol  contamination 
and  one  group  is  free  from  aerosol  contamination.  The  canonical  variates  are  then 
used  to  discriminate  between  the  groups.  Separate  canonical  variate  functions  have 
been  computed  for  day  versus  night  retrievals  and  for  different  geographic  areas. 

Let  B  be  the  between  group  covariance  matrix  and  W  the  within  group  covari¬ 
ance  matrix.  Linear  variate  functions  (/)  are  found  to  maximize, 


v  =  X'BX/X'WX 


(4.2) 


which  represent  the  ratio  of  between  group  to  within  group  variances.  Differentiat¬ 
ing  Eq.  (4.2)  and  setting  to  zero  gives, 


0 B  -  vW)X  =  0. 


(4.3) 


The  eigenvalues  of  W  ‘B  and  the  corresponding  eigenvectors  (i)  are  the  canonical 
variate  functions  used  for  discrimination.  The  observed  channel  brightness  tem¬ 
peratures  and  NAAPS  wavelength  dependent  AOD  components  are  projected  onto 
the  canonical  variates  and  the  Euclidean  distances  to  the  projected  group  means 
is  determined.  The  SST  retrieval  is  classified  as  contaminated  if  the  distance  to  a 
contaminated  group  mean  is  closer  than  the  distance  to  the  non-contaminated  group 
mean  according  to, 


r 


(4.4) 


where  x  is  the  AOD  wavelength  components  and  AVHRR  channel  brightness  tem¬ 
peratures  vector  for  a  given  SST  retrieval,  r  is  the  number  of  canonical  variate  func¬ 
tions,  is  the  group  mean  vector  of  observed  values  for  the  contaminated  and  non- 
contaminated  groups,  and  k  is  the  group  classification  code  The  group  assignment 
probability  is  computed  assuming  that  group  distances  are  chi-square  distributed 
with  r-1  degrees  of  freedom.  Satellite  SST  retrievals  assigned  to  a  contaminated 
aerosol  group  can  either  be  flagged  for  rejection  or  corrected  using  radiative  trans¬ 
fer  modeling  that  takes  into  account  the  height  distribution  of  the  aerosol  plume  in 
NAAPS  and  the  vertical  distribution  of  temperature  from  an  atmospheric  forecast 
model  (Merchant  et  al,  2006). 

4.4. 4.3  Diurnal  Warming 

Surface  diurnal  warming  events  are  common  in  the  world  oceans.  The  warming 
events  produce  near-surface  thermal  gradients  that  create  daytime  near-surface  or 
warm-layer  temperatures  2 -4°C  warmer  than  nighttime  (Donlon  et  al.  2002).  Al¬ 
though  not  strictly  a  measurement  error,  combining  SST  measurements  with  dif¬ 
ferent  observation  times  in  a  daily  analysis  requires  consideration  of  diurnal  warm¬ 
ing  events.  Knowledge  of  diurnal  warming  events,  in  turn,  requires  information  on 
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the  local  time  history'  of  the  wind  speed  and  surface  solar  radiation  at  the  time  of 
the  SST  observation.  However,  often  only  instantaneous  measures  of  surface  wind 
speed  and  solar  radiation  fields  from  NWP  systems  are  collocated  with  satellite 
SST  retrievals.  Nevertheless,  detection  of  diurnal  wanning  and  potential  skin-layer 
effects  in  satellite  SST  retrievals  is  still  possible  given  the  presence  of  low  winds, 
high  solar  insolation,  and  a  positive,  statistically  significant  change  in  SST  from  a 
background  field  valid  w  ithin  ~6  hours  of  the  observation  time. 


4.4.4.4  Microwave  SST 

Due  to  lower  signal  strength  of  the  radiation  curve  in  the  microwave  region,  accuracy 
and  resolution  is  poorer  for  SST  derived  from  passive  microwave  measurements  as 
compared  to  SST  derived  from  thermal  infrared  measurements.  However,  the  advan¬ 
tage  gained  with  passive  microwave  is  that  radiation  at  these  longer  wavelengths  is 
largely  unaffected  by  clouds  and  generally  easier  to  correct  for  atmospheric  effects. 
Phenomena  which  do  affect  passive  microwave  signal  return,  however,  are  wind¬ 
generated  roughness  at  the  ocean  surface  and  precipitation.  These  affects  can  usually 
be  corrected  for  using  multiple  frequencies.  SST  measurements  are  primarily  made 
at  a  channel  near  7  GHz  with  a  water  vapor  correction  enabled  by  observations  at 
21  GHz.  Other  frequencies  used  for  correction  of  surface  roughness  (including  foam), 
precipitation,  and  what  little  effect  clouds  do  have  on  microwave  radiation,  include 
information  in  the  11,  18,  and  37  GHz  channels.  Nevertheless,  rain  contamination 
continues  to  be  a  problem  at  the  edge  of  rain  cells,  where  there  is  often  undetected  rain 
that  cause  biased  SST  retrievals.  Land  contamination  is  also  an  issue  with  microwave 
measurements.  Within  50-100  km  of  land  microwave  measurements  are  affected  by 
emissions  from  land  resulting  in  a  warm  bias  in  coastal  microwave  SST.  For  this  rea¬ 
son,  microwave  SST  observations  are  typically  not  produced  within  100  km  of  land. 


4.4.5  Sea  Ice  Concentration 

A  problem  with  sea  ice  concentration  retrievals  from  the  SSM/1  and  SSMIS  sensors 
on  board  the  DMSP  series  of  satellites  is  false  indication  of  sea  ice  over  the  open 
ocean  and  at  the  ice  edge.  These  spurious  sea  ice  concentrations  result  from  the 
presence  of  atmospheric  water  vapor,  non-precipitating  cloud  liquid  w  ater,  rain,  and 
sea  surface  roughening  by  surface  winds.  While  these  effects  are  relatively  minor 
at  polar  latitudes  in  winter,  they  result  in  serious  weather  contamination  problems 
at  all  latitudes  in  summer.  The  various  sea  ice  retrieval  algorithms  used  operation¬ 
ally  attempt  to  eliminate  these  false  positive  sea  ice  concentrations,  but  with  limited 
success.  Accordingly,  prior  to  assimilation,  sea  ice  concentrations  need  to  be  qual¬ 
ity  controlled  using  a  weather  filter.  A  good  proxy  for  a  weather  filter  is  SST.  If 
the  sea  ice  concentration  is  greater  than  zero  and  the  collocated  SST  exceeds  4°C, 
then  it  is  likely  the  sea  ice  retrieval  is  contaminated  by  a  weather  event  and  should 
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be  rejected.  The  4°C  SST  threshold  is  considered  to  be  very  conservative.  Tests 
with  a  1°C  SST  threshold  resulted  in  spunous  rejections  of  sea  ice  retrievals  in  the 
East  Greenland  Current  during  periods  of  rapid  ice  growth,  when  the  analyzed  SST 
fields  are  not  accurate  due  to  a  lack  of  SST  observations.  A  cross  validation  check 
will  not  work  as  a  weather  filter,  since  weather  contamination  is  typically  large 
scale  affecting  many  near-by  sea  ice  retrievals  simultaneously. 

Sea  ice  retrieval  algorithms  also  return  false  positive  ice  conditions  near  land  due 
to  land  contamination  of  the  microwave  signal.  This  bias  is  most  evident  during  the 
summer  ice  melt  season  in  the  northern  hemisphere  when  the  Arctic  land  boundar¬ 
ies  become  ice  free.  A  high  resolution  distance  from  land  database  is  used  to  check 
if  the  retrieval  is  within  100  km  of  land.  The  test  uses  retrieval  distance  from  land, 
background  field  anomalies,  collocated  SST,  and  a  cross  validation  check  of  nearby 
locations  to  determine  if  positive  sea  ice  observations  near  land  are  valid.  Land 
contaminated  sea  ice  retrievals  are  typically  rejected  at  this  point. 


4 . 4. 6  Temperature  and  Salinity  Profiles 

Profile  observations  are  first  checked  for  duplicate  depths  and  strictly  increasing 
depths.  Reported  levels  that  fail  these  tests  are  flagged  and  not  used  in  the  following 
profile  quality  control  procedures. 


4.4. 6.1  Instrumentation  Error  Checks 

Special  instrument  specific  error  tests  are  applied  to  profile  observations  to  identify 
errors  that  have  unique  profile  signatures.  These  errors  include  temperature  inver¬ 
sions  at  the  bottom  of  the  profile,  spikes  in  the  temperature  profile,  and  positive  tem¬ 
perature  gradients  (warm  bulge)  in  the  mixed  layer.  The  instrumentation  error  checks 
are  applied  iteratively  until  all  errors  are  found,  since  a  profile  may  have  one  or  more 
of  these  types  of  errors.  Reported  temperature-depth  levels  that  contain  instrumenta¬ 
tion  errors  are  flagged  and  not  used  in  the  next  iteration  of  the  instrumentation  error 
checks.  One  difficulty  with  the  current  suite  of  profile  instrumentation  error  checks 
is  that  the  tests  are  designed  to  detect  errors  specific  to  expendable  bathythermo¬ 
graphs  (XBT)  (Bailey  et  al.  1994).  Other  profile  data  types,  such  as  Argo  floats, 
gliders  and  CTD  probes,  are  likely  to  have  failure  modes  that  are  different  from  a 
XBT.  Automated  quality  control  tests  need  to  be  developed  to  detect  instrumentation 
errors  in  these  data  types  as  more  experience  is  gained  with  their  assimilation. 


4.4. 6.2  Static  Stability 

A  static  stability'  test  is  performed  to  detect  density  inversions  in  profile  observa¬ 
tions.  The  reported  in  situ  temperature  and  depth  data  pairs  are  first  converted  to 
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potential  temperature  and  pressure,  and  then  potential  density  is  computed  at  each 
pressure  level  using  observed  or  derived  salinity  values.  Salinity  observations  are 
generated  for  profiles  that  report  only  temperature.  Salinity  is  computed  from  ob¬ 
served  temperature  values  using  bi-monthly  climatological  temperature-to-salinity 
regression  models  that  have  been  computed  on  a  global  0.25°  resolution  grid.  The 
potential  density  profile  is  examined  for  inversions  (higher  density  shallower  than 
lower  density),  and  observed  temperature  and  salinity  profile  levels  with  inversions 
that  exceed  a  minimum  specified  inversion  threshold  of  0.025  kg  m  *3  are  flagged. 
For  profiles  with  derived  salinities,  static  instabilities  are  corrected  by  iteratively 
adjusting  the  derived  salinity  until  the  resulting  profile  is  neutrally  buoyant.  Salinity 
is  removed  from  the  top  of  the  permanent  thermocline  upward  and  added  from  that 
depth  downward  in  the  adjustment,  llie  salinity  correction  algorithm  is  not  applied 
to  density  inversions  for  profiles  that  observe  both  temperature  and  salinity  levels, 
since  it  is  difficult  to  determine  a  priori  if  the  cause  of  the  density  inversion  is  due 
to  the  reported  temperature  or  the  reported  salinity  value.  In  this  case  profile  levels 
with  density  inversions  are  simply  flagged. 


4.4. 6.3  Vertical  Gradient  Checks 

A  global  climatology  of  vertical  mean  temperature  differences  and  standard  de¬ 
viations  about  these  means  has  been  computed  from  the  historical  profile  ar¬ 
chive.  The  climatology  is  used  to  test  observed  vertical  temperature  gradients 
for  outliers.  First,  the  climate  temperature  differences  and  variability  are  inter¬ 
polated  to  the  observation  location  and  sampling  time.  Second,  the  vertical  tem¬ 
perature  differences  are  converted  to  vertical  temperature  gradients  and  inter¬ 
polated  to  the  observed  profile  levels.  Observed  vertical  temperature  gradients 
are  computed,  and  the  difference  between  the  observed  and  the  expected  mean 
vertical  gradient  from  the  climatology  is  standardized  by  the  expected  gradient 
variability, 


z  =  (A  T„  -  m  1 -A  Tc-m  l)/a 


(4.5) 


where  ATo  •  m  1  is  the  observed  vertical  gradient,  AT,  m  1  is  the  climate  mean 
vertical  gradient,  a  is  the  variability  about  that  mean,  and  z  is  the  standardized  verti¬ 
cal  gradient  variate.  If  the  observed  profile  gradient  exceeds  0.2°C*m  1  and  |z|>4, 
then  the  profile  level  is  flagged.  Experience  has  shown  that  the  vertical  gradient  test 
based  on  climate  statistics  tends  to  spuriously  flag  as  erroneous  profile  levels  as¬ 
sociated  with  a  strong  thermocline.  This  problem  is  particularly  acute  in  the  tropics. 
Truly  erroneous  profile  vertical  gradients  are  often  associated  with  bad  temperature 
or  salinity  observations,  which  are  detected  in  the  spike  test  and  background  field 
checks  described  previously.  Hence,  at  the  present  time,  quality  control  flags  set  by 
the  vertical  gradient  check  are  flagged  and  only  used  for  informational  purposes 
(Sect.  5). 
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4,4. 6.4  Profile  Shape  Comparisons 

Observed  profiles  are  compared  to  profiles  extracted  from  the  various  background 
fields  using  a  profile  shape  quality  control  procedure.  This  procedure  has  the  advan¬ 
tage  of  taking  an  overview  of  the  entire  profile.  Profile  levels  that  have  previously 
been  determined  to  be  unreliable  based  on  other  profile  quality  control  data  checks 
are  excluded  in  the  profile  shape  quality  control  procedure.  The  shape  quality  con¬ 
trol  procedure  computes  an  integrated  observed-minus-predicted  statistic  that  takes 
into  account  level  thicknesses.  The  test  statistic  is  calculated  as, 

n  =  ^((0*  -  Pk)/°k)  •  (Zk+i  -  z*-i)/£(z*+i  -  tk-i )  (4.6) 

k  k 

where  Ok  is  the  observed  value  at  level  k,  Pk  is  the  prediction  (background)  value  at 
level  k,  ak  is  the  prediction  error  standard  deviation  at  level  k,  and  zk  is  the  depth  of 
level  k.  The  probability  of  //  being  greater  than  zero  is  computed  assuming  a  normal 
probability  distribution  function.  The  shape  comparison  statistic  is  analogous  to  a 
goodness-of-fit  test  of  two  cumulative  distribution  functions.  It  identifies  observed 
profiles  with  large  errors  relative  to  the  background  profiles.  Profiles  that  have  large 
temperature  or  salinity  differences  over  narrow  depth  ranges,  such  as  dissimilar 
mixed  layer  depths,  will  be  considered  similar.  Observed  profile  shape  must  be 
consistent  with  forecast  and  climate  background  profiles  in  order  for  the  profile  to 
be  accepted  into  the  analysis. 


4.4. 6.5  Gliders 

Ocean  gliders  are  autonomous  platforms  which  fly  in  a  saw-tooth-sampling  pat¬ 
tern  in  the  upper  ocean  by  changing  their  buoyancy.  Depending  upon  configura¬ 
tion,  gliders  sample  profiles  of  pressure,  temperature,  and  conductivity.  The  gliders 
surface  at  regular  intervals  to  transmit  their  observations  to  shore  or  satellite  based 
receivers.  Gliders  provide  both  downward  and  upward  profiles  of  temperature  and 
salinity,  with  glider  position  and  time  varying  with  depth  during  the  dive.  Quality 
control  of  glider  data  is  similar  to  that  of  single  profile  data,  other  than  relaxation 
of  the  strictly  increasing  depths  check.  However,  several  glider  specific  tests  are 
performed  that  are,  in  most  cases,  functions  of  the  vertical  velocity  of  the  glider 
These  tests  are  applied  to  gliders  with  a  non-pumped  CTD,  where  flow  through  the 
conductivity  cell  depends  upon  the  speed  of  the  glider,  making  the  thermal  inertial 
correction  speed-dependent. 


4.4. 7  Altimeter  Sea  Surface  Height 

The  along-track  altimeter  data  undergo  an  extensive  series  of  pre-processing  steps 
to  prepare  the  data  for  use  in  the  assimilation.  The  measured  sea  surface  height 
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(SSH)  is  corrected  for  geophysical  effects  (wet  and  dry  troposphere,  ionosphere, 
inverted  barometer,  and  winds),  and  the  tidal  signal  is  removed.  The  corrected  SSH 
from  each  satellite  altimeter  mission  are  then  intercalibrated  with  a  global  crossover 
adjustment  using  Topex/Poseidon  data  as  the  reference.  Next,  the  data  are  resa¬ 
mpled  every  7  km  (1  sec  intervals)  along  the  tracks.  A  mean  SSH  is  removed  from 
the  individual  SSH  measurements  producing  sea  surface  height  anomalies  (SSHA). 
The  mean  SSH  contains  both  the  unknown  geoid  signal  and  the  mean  dynamic 
topography  over  the  averaging  period.  For  most  satellite  missions  a  mean  SSH  cal¬ 
culated  over  a  7-year  period  is  used,  although  the  averaging  period  continues  to  be 
extended  in  time  as  the  altimeter  satellite  missions  continue.  These  altimeter  pre¬ 
processing  steps  are  typically  performed  by  the  data  provider. 

Altimeter  SSHA  observations  are  of  lower  accuracy  or  are  not  interpretable  near 
the  coasts  due  to  inaccurate  tidal  corrections  and  incorrect  removal  of  atmospheric 
wind  and  pressure  effects  at  the  sea  surface  in  shallow  water.  The  coastal  region  for 
altimeter  data  assimilation  is  often  defined  as  everywhere  shallower  than  400  m 
depth.  Altimeter  observations  also  have  significant  along-track  correlated  errors 
that  must  be  taken  into  account  in  the  assimilation.  The  along-track  altimeter  SSHA 
data  are  very  noisy  at  the  full  7  km  resolution  Accordingly,  altimeter  SSHA  data 
are  smoothed  along-track  using  a  median  or  Lanczos  filter  to  reduce  the  measure¬ 
ment  noise.  In  addition,  the  altimeter  data  are  often  sub-sampled  or  bin-averaged  to 
remove  redundant  observations.  Finally,  altimeter  SSHA  measurements  are  scaled 
by  a  hyperbolic  tangent  operator  using  local  dynamic  height  variability  limits  that 
have  been  computed  from  the  historical  profile  archive.  This  operation  attempts  to 
remove  spurious  altimeter  SSHA  outliers  and  maintain  the  data  within  the  range  of 
known  baroclinic  variability  limits. 

A  final  issue  with  altimeter  SSHA  measurements  is  the  fact  that  different  ver¬ 
sions  of  the  data  are  reported  in  both  near  real-time  and  in  delayed  mode.  Real-time 
SSHA  observations  are  computed  using  less  precise,  predicted  orbits  rather  than 
the  more  precise,  observed  orbits,  which  are  not  available  for  several  days  after 
real-time.  Although  less  precise,  real-time  SSHA  observations  still  have  significant 
value  in  the  analysis.  However,  when  the  more  precise  delayed  mode  SSHA  ob¬ 
servations  are  available,  the  corresponding  real-time  SSHA  data  should  be  identi¬ 
fied  and  replaced  by  the  delayed  mode  data.  This  procedure  ensures  that  the  higher 
quality,  delayed  mode  SSH  observations  are  incorporated  into  the  altimeter  SSHA 
data  archive  for  use  in  hindcast  studies.  Satellite  altimeter  SSHA  observations  are  a 
critical  data  source  in  GODAE  assimilation  systems  and  timely  access  to  the  most 
complete,  highest  quality  data  is  essential. 


4.4.8  Altimeter  Significant  Wave  Height 

Comparisons  with  buoy  data  show  that  altimeter  SWH  estimates  are  in  agreement 
with  the  in  situ  data,  with  standard  deviations  of  differences  on  the  order  of  0.30  m, 
but  the  satellite  data  tends  to  slightly  overestimate  low  SWH  and  slightly  underes¬ 
timate  high  SWH.  The  altimeter  SWH  data  thus  needs  to  be  bias  corrected  before 
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being  used  in  the  assimilation.  These  bias  corrections  are  generally  linear  and  are 
derived  from  altimeteribuoy  matchups  that  correspond  to  corrections  of  only  a  few 
percent  of  SWH.  Altimeter  SWH  data  can  also  be  contaminated  by  sea  ice  or  land. 
Elimination  of  these  data  requires  a  contemporaneous  sea  ice  concentration  field,  ei¬ 
ther  from  a  forecast  model  or  an  analysis  of  SSM/I  and  SSMIS  sea  ice  retrievals.  The 
land  mask  needs  to  resolve  the  along-track  7-km  footprint  of  the  altimeter  SWH  data. 


4.5  Quality  Control  Decision-Making  Algorithms 

The  quality  control  outcomes  of  the  various  external  data  checks  described  previ¬ 
ously  are  combined  in  a  decision-making  algorithm.  The  outcome  of  the  decision¬ 
making  algorithm  is  the  overall  indication  of  observation  quality,  which  is  used  to 
select  data  for  the  assimilation.  The  decision-making  algorithm  is  applied  to  each 
observed  reporting  level  and,  in  the  case  of  profile  (and  glider)  observations,  to  the 
entire  profile  in  the  shape  comparison  test.  Thus,  for  profile  observations  there  are 
two  indicators  of  data  quality  :  one  indicator  for  the  overall  profile  shape,  and  a  sec¬ 
ond  indicator  for  each  profile  level.  It  is  important  to  take  into  consideration  results 
from  all  of  the  external  data  checks  before  the  final  quality  decision  is  made.  For 
example,  an  observation  could  fail  the  climate  background  check  while  at  the  same 
time  pass  the  forecast  background  check.  The  observation  would  be  rejected  if  the 
climate  test  was  applied  first  in  a  serial  fashion.  Quality  control  decision-making 
algorithms,  therefore,  are  necessarily  complex  and  must  combine  outcomes  from 
the  different  external  test  results  appropriately.  It  helps  if  the  external  test  outcomes 
are  of  the  same  form,  such  as  probabilities  or  standard  normal  deviates. 

A  quality  control  decision-making  algorithm  in  use  at  the  U.S.  Navy  oceano¬ 
graphic  centers  is  described  here.  The  quality  control  outcomes  from  the  various 
external  data  checks  are  in  the  form  of  probabilities  of  error  The  majority  of  these 
probabilities  are  calculated  according  to  Eq.  (4.1),  assuming  a  normal  probability 
density’  function,  but  probabilities  are  also  calculated  using  chi-square  distribution 
functions  (i.e.,  aerosol  contamination  test).  Given  a  set  of  error  probabilities  the 
decision-making  algorithm  is  summarized  as  follows: 

Pb  =  min  (Pg,  Pr) 

Pd  =  min(/>c,  Px) 

Pb<XfiPo  =  pb 

Pb  >  T/,  P0  =  min  ( Pb ,  Pd) 

where  pb  is  the  composite  background  error  probability,  pd  is  the  composite  data- 
derived  error  probability,  pg  and  pr  are  the  global  and  regional  forecast  background 
error  probabilities,  pc  and  px  are  the  climate  and  cross  validation  error  probabilities, 
rf  is  the  forecast  error  threshold  probability,  and  po  is  the  overall  probability  the 
observation  contains  a  random  error.  The  forecast  error  probability  threshold  for  the 
system  is  typically  set  to  0  99  (3  standard  deviations). 

The  algorithm  first  determines  if  the  observation  is  consistent  with  the  model 
background  fields  by  taking  the  minimum  error  probability  of  the  global  and  re- 
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gional  forecasts.  If  the  minimum  background  error  probability  is  less  than  the  pre¬ 
scribed  forecast  error  tolerance  limit,  then  the  algorithm  returns  it  as  the  overall 
probability  of  error  for  the  observation.  However,  if  the  minimum  model  back¬ 
ground  error  probability  exceeds  the  forecast  error  threshold,  then  it  is  compared 
against  the  data-derived  error  defined  as  the  minimum  of  the  cross  validation  and 
climatology  error  probabilities.  The  overall  observation  error  probability  is  returned 
as  the  minimum  of  the  composite  background  and  composite  data-derived  errors. 
In  this  way,  cross  validation  and  climate  backgrounds  determine  data  quality  only 
if  the  observation  is  not  consistent  with  the  forecast.  Experience  has  shown  that 
requiring  observations  to  always  be  consistent  with  climate  backgrounds  results  in 
spurious  rejection  of  valid  observations  during  extreme  events. 

Once  the  overall  probability  of  error  for  an  observation  has  been  determined, 
output  from  the  various  specific  observing  system  quality  control  tests  are  simply 
added  to  the  error  probability  using  unique  integer-valued  flags.  The  quality  con¬ 
trol  flags  have  three  levels  of  severity:  (1)  information-only  (<100);  (2)  cautionary 
(>100);  and  (3)  fatal  (>1,000).  Observations  with  fatal  errors  are  not  used  in  the 
analysis.  Information-only  flagged  observations  are  routinely  used  in  the  analysis, 
but  the  use  of  cautionary  flagged  observations  is  under  user  control  via  analysis 
namelist  options.  The  ultimate  decision  to  accept  an  observation  into  the  analysis, 
however,  is  always  based  on  the  underlying  error  probability  value  obtained  from 
the  decision-making  algorithm  If  quality  control  flags  have  been  appended,  the 
underlying  probability  of  error  can  always  be  recovered  from  the  summation  using 
some  simple  modular  arithmetic. 


4. 5. 1  Quality  Control  System  Performance 


Output  from  the  U.S.  Navy’s  fully  automated  real-time  ocean  data  control  system 
is  summarized  for  satellite  SST  retrievals,  sea  ice  concentration  retrievals,  altimeter 
sea  surface  height  and  significant  wave  height  retrievals,  and  in  situ  observations  at 
the  surface  and  at  depth  from  various  sources.  Quality  control  output  for  the  satel¬ 
lite  data  is  given  for  two  monthly  time  periods  during  2009  (June  and  December)  to 
allow  for  examination  of  possible  effects  of  seasonality,  while  output  from  quality 
control  of  the  in  situ  data  is  shown  for  the  entire  2009  year.  The  overall  quality  of 
the  observations  is  summarized  using  an  error  probability  frequency  of  occurrence 
in  per  cent.  The  error  probabilities  are  the  outcomes  of  the  quality  control  decision¬ 
making  algorithm  for  single  level  observations  and  the  overall  probability  of  er¬ 
ror  for  profile  observations.  Assuming  a  normal  probability  distribution  function, 
the  frequency  of  occurrence  bins  correspond  to  one  standard  deviation  (p<0.67), 
two  standard  deviation  (p<0.95),  and  three  standard  deviation  (p<0,99)  departures 
from  a  zero  mean.  Probability  frequencies  indicated  as  p<  1.0  include  probabilities 
greater  than  0.99  plus  observations  flagged  as  being  suspect  from  one  or  more  of  the 
specific  external  data  checks  described  previously.  Observations  with  error  prob¬ 
abilities  less  than  0.99  are  typically  accepted  into  the  analysis. 
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In  general,  QC  outcomes  of  the  satellite  SST  retrievals  indicate  that  the  data  are 
of  good  quality  (Table  4.1).  The  frequencies  of  error  probabilities  within  one  stan¬ 
dard  deviation  of  the  background  field  consistently  include  90%  or  more  of  the  data 
for  all  satellite  systems.  Allowing  for  two  background  error  standard  deviations  re¬ 
sults  in  more  than  -99%  of  the  observations  being  included.  There  is  some  evidence 


Table  4.1  Real-time  QC  outcomes  for  satellite  SST  retrievals 


Satellite 

Month 

2009 

Type 

Countx 

106 

Diurnal 

Aerosol1 

p<0.67 

p<0.95 

p<0.99 

p<  1.0 

jMSR-e 

Jun 

- 

87.82 

- 

96.2 

3.7 

0  1 

0.1 

Dec 

Day 

47.68 

23,427 

- 

94.5 

5.3 

0.2 

0.1 

Night 

55.59 

- 

- 

95.5 

4.3 

0.1 

0.0 

AATSR2  3 4 

Jun 

Day 

220.35 

364,910 

30,656 

93  0 

6.3 

0,5 

0.3 

Night 

330.58 

- 

195,971 

91.2 

8.4 

04 

0  1 

Dec 

Day 

230.32 

161,863 

8,391 

95  0 

4  7 

0  2 

0  1 

Night 

317.16 

- 

42,313 

91.9 

7.6 

0.4 

0.0 

GOES-II 

Jun 

Day 

26.93 

258 

12 

89.8 

10.1 

0.1 

0.0 

Night 

70.84 

- 

4 

95  2 

4.7 

0.1 

0  0 

Dec 

Day 

37,67 

97.6 

2.3 

0.0 

0.0 

Night 

88.80 

95.8 

4  1 

0.1 

0.0 

GOES-12 

Jun 

Day 

19.06 

1,043 

7,083 

96.7 

3.2 

0.1 

00 

Night 

53.33 

- 

435,078 

93.3 

5.7 

0.2 

0.8 

Dec 

Day 

27.44 

1,014 

49 

95.4 

4.6 

00 

0.0 

Night 

66  30 

- 

12,519 

93.1 

6.7 

0  2 

00 

METOP 

Jun 

Day 

5.46 

938 

2,541 

976 

2.3 

0.1 

0.1 

GAC 

Night 

5.63 

- 

5,462 

94.7 

5.0 

0.2 

0.1 

Dec 

Day 

6.09 

862 

35 

97.5 

2.4 

0.1 

0.0 

Night 

5.89 

- 

144 

95.4 

4.4 

02 

0.0 

METOP 

Jun 

Day 

106.52 

28,165 

86,935 

96.2 

3.5 

0.1 

0.1 

LAC 

Night 

119.47 

- 

44,456 

95.5 

4.4 

0.2 

0.0 

Dec 

Day 

216.67 

20,350 

3,312 

97.4 

2.5 

0  1 

0.0 

Night 

234.74 

- 

9,060 

94.5 

5.3 

0.2 

0.0 

MSG 5 

Jun 

Day 

14.47 

2,995 

10,202 

94.8 

4.5 

0.4 

0.3 

Night 

73.28 

- 

13,343 

94.8 

4.8 

0.2 

0.2 

Dec 

Day 

12.23 

25,999 

759 

95.3 

4.2 

0.2 

0.3 

Night 

11.55 

- 

3,082 

94.9 

4.9 

0.2 

0.0 

NOAA-18 

Jun 

Day 

4.71 

148 

14 

90.7 

8.6 

06 

0.1 

Night 

5  24 

- 

5,072 

95.3 

4.4 

0.2 

0.1 

NOAA-19 

Dec 

Day 

5.08 

11,919 

36 

88.9 

10.2 

0.6 

0.3 

Night 

4  99 

298 

95.4 

4.4 

0.3 

0.0 

1  Aerosol  contamination  calculated  for  Saharan  Dust  events  in  an  area  bounded  by  10°S-30°N, 
25°E-55°W 


2  AMSR-E  not  partitioned  into  day/night  retrievals  in  June.  AMSR-E  data  missing  16-17  June 
067,  1 8  June  00-12Z,  20  June,  23  June,  25-26  June,  28-30  June,  29  Dec  12-247 

3  AATSR  data  missing  16  June  00-067,  20  June  00-067,  26  June  00-187,  28  June  00-127,  29 
June  12-187.  8  Dec  00-067,  24  Dec  06-127 

4  METOP  LAC  data  missing  19  Dec  00-  127;  27  Dec  18-247 

5  MSG  data  missing  6  June  12-18Z,  13  June  00-067,  15  June  00-067,  16-18  June,  20-21  June 
007,  22  June  06-1 2Z,  23-24  June  00Z,  25-30  June,  15  Dec  06-127 
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Table  4.2  Real-time  QC  outcomes  for  satellite  sea  ice  retrievals 


Satellite1 

Month  2009 

Count  x  106 

Weather 

Filter2 

p<0.67 

p<0  95 

p<0.99 

p<1.0 

FI3 5 

Jun 

5.23 

570 

97.1 

2.1 

0.5 

0.3 

Dec 

- 

- 

- 

- 

- 

- 

FI5 

Jun 

10.65 

2,777 

96  2 

2.6 

0.7 

0.5 

Dec 

11.63 

1,048 

94.5 

3.6 

l.l 

0.8 

FI6 

Jun 

16.78 

17,070 

96  5 

2.4 

0.6 

0.5 

Dec 

18.32 

3,478 

95.3 

3.3 

0.9 

0.6 

F17 

Jun 

16.64 

13,687 

97  1 

2.1 

0.4 

0.3 

Dec 

18.87 

3,652 

95.5 

3.2 

0.8 

0.5 

Shelf  Ice 

Jun 

065 

- 

77.6 

104 

5  8 

6  1 

Dec 

0.44 

- 

74.7 

16.7 

5.7 

2.9 

1  F 13  and  FI  5  are  SSM/I  satellites;  F16  and  FI  7  are  SSMI/S  satellites 

2  Weather  filter  based  on  collocated  analyzed  SST  values  (see  text  for  details) 

3  F 13  data  use  discontinued  in  December 


of  seasonality  in  the  number  of  retrievals  detected  as  coming  from  diurnal  warming 
and  aerosol  contamination  events  for  AATSR,  GOES,  METOP  and  MSG  data.  Sea 
ice  concentration  retrievals  from  the  SSMI  and  SSMI/S  satellites  are  also  of  good 
quality:  -99%  of  the  data  fall  within  two  standard  deviations  of  the  background 
field  (Table  4.2).  The  number  of  sea  ice  retrievals  rejected  by  the  weather  filter 
based  on  collocated  SST  shows  a  clear  seasonality  with  many  more  weather  filter 
rejections  in  June  than  in  December.  Altimeter  sea  surface  height  (SSH)  observa¬ 
tions  are  also  of  good  quality  with  -99%  of  the  data  within  two  standard  deviations 
(Table  4.3).  Altimeter  significant  wave  height  (SWH)  observations  appear  to  be  of 
lower  quality,  but  SWH  rejections  are  mostly  over  land  or  ice  covered  seas  (defined 
here  as  33%  sea  ice  concentration).  Quality  control  of  altimeter  SWH  retrievals  is 
model  based  in  the  Navy  system.  A  6-hour  forecast  from  a  data  assimilative  run 
of  the  wave  model  is  used  to  check  newly  received  altimeter  and  buoy  SWH  ob¬ 
servations  for  consistency,  ensuring  that  the  valid  time  of  the  forecast  corresponds 
closely  to  the  observed  times  of  the  data. 

Table  4.4  gives  QC  outcomes  for  in  situ  SST  observations  from  ships  and  buoys. 
Ship  data  are  of  lower  quality  than  buoy  data,  with  about  8%  of  the  ship  data  being 
rejected  across  the  different  ship  data  types.  Drifting  buoy  data  are  of  higher  quality 
than  fixed  buoys,  with  fixed  buoy  data  showing  increased  variability  as  indicated 
by  the  large  percentage  of  data  in  the  probability  range  of  0.67-0.95.  Profile  data 
QC  is  summarized  in  Table  4.5.  Recall  that  profile  levels  with  density  inversions 
or  vertical  gradient  information-only  flags  do  not  affect  use  of  those  data  in  the 
assimilation.  The  large  number  of  TESAC  data  is  a  result  of  fixed  buoys  report¬ 
ing  both  temperature  and  salinity  using  the  WMO  TESAC  code  form.  These  data 
report  only  a  single  or  very  few  vertical  levels  and  are  of  low  quality,  with  less  than 
75%  of  the  data  occurring  within  two  standard  deviations  of  the  background  field 
XBT  observations  have  large  occurrences  of  vertical  gradient  and  instrumentation 


Table  4.3  Real-time  QC  outcomes  for  satellite  altimeter  retrievals 
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Table  4,4  Real-time  QC  outcomes  for  in  situ  surface  temperature  observations  in  2009 


Type 

Count  x  103 

p<0.67 

p<0.95 

p<0.99 

p<1.0 

Ship  ERJ 

210.3 

55.5 

27.0 

9.0 

8.5 

Ship  Bucket 

32.1 

47.2 

31.4 

12.6 

8.8 

Ship  Hull  Contact 

309.2 

53.6 

28.5 

10.2 

7.7 

CM  AN  Station 

23.6 

72.1 

20.6 

5.0 

2.2 

Fixed  Buoy 

2,657.3 

83.5 

13,3 

2.6 

0.7 

Drifting  Buoy 

10,624.1 

92.3 

5.8 

09 

1.0 

Table  4.5  Real-time  QC  outcomes  for  profile  observations  in  2009 


Type 

Count1  x  Density 
103  lnv.2 

Vertical 

Grad.2 

Inst. 

Error2-3 

Depth  Missing 
Error2  Value2-4 

p<0.67 

p<0.95 

p<0.99 

p<  1.0 

XBT 

189 

- 

12,722 

52,301 

674 

26 

75.9 

16.2 

1.6 

6.3 

Fixed 

502.5 

19,000 

3,922 

- 

- 

1,163 

81.3 

16.3 

1.5 

0.9 

Buoy 

Drifting 

31.7 

207 

5,743 

6,374 

- 

- 

84.3 

8.1 

1.9 

5.7 

Buoy 

TESAC 

1,332.4 

1,382 

2,165 

1,706 

551 

222 

440 

29.3 

10.0 

16.7 

Argo 

148.2 

9,028 

8,801 

6,669 

4,628 

7,158 

77.9 

18.3 

1.7 

2.1 

1  Counts  are  number  of  profiles 

2  Counts  are  number  of  profile  lev  els  affected 

3  Instrumentaton  error  includes  wire  stretch,  wire  breaking,  invalid  upper  ocean  temperature 
response,  profile  spikes 

4  Counts  refer  to  missing  temperature  levels  only 


errors,  which  are  probably  due  to  inflexion  point  decimation  of  the  profiles  done 
prior  to  posting  the  data  on  the  GTS.  Argo  is  of  high  quality  with  more  than  96% 
of  the  profiles  accepted  into  the  analysis.  However,  Argo  profiles  show  a  relatively 
high  occurrence  of  depth  errors  (duplicate  depths  or  depths  not  strictly  increasing) 
and  missing  value  errors  (defined  here  in  terms  of  temperature)  that  need  to  be 
investigated. 


4.6  Internal  Data  Checks 

Internal  checks  are  those  quality  control  procedures  performed  by  the  analysis  sys¬ 
tem  itself.  These  data  consistency  checks  are  best  done  within  the  assimilation  algo¬ 
rithm  since  it  requires  detailed  knowledge  of  the  background  and  observation  error 
covariances,  which  are  available  only  when  the  assimilation  is  being  performed 
The  internal  data  checks  are  the  last  defense  of  the  assimilation  algorithm  against 
bad  observations.  Data  that  contain  gross  and  random  errors  have  hopefully  been 
removed  prior  to  the  assimilation  in  the  sensibility  and  external  data  checks.  The 
purpose  of  the  internal  data  checks  is  to  decide  whether  any  marginal  observations 
remaining  in  the  assimilation  data  set  are  acceptable  or  unacceptable. 
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The  need  for  quality  control  at  this  stage  of  the  analysis/forecast  system  cannot 
be  over  emphasized.  Any  assimilation  system  based  on  the  assumption  of  normality, 
no  matter  how  sophisticated,  is  vulnerable  to  bad  observations  that  do  not  fit  a  nor¬ 
mal  distribution.  Further,  since  many  GODAE  forecasting  systems  use  a  sequential 
analysis-forecast  cycle,  it  is  difficult  to  remove  the  propagation  of  error  through  the 
forecast  period  that  occurs  when  erroneous  data  have  been  assimilated.  Once  this 
happens  the  only  option  is  to  blacklist  the  bad  observations  and  back-up  and  rerun 
the  analysis-forecast  cycle.  This  remedy  will  cause  a  delay  in  the  production  of  the 
forecast,  which  can  be  a  serious  problem  in  operations  since  the  forecast  products 
are  time  critical. 

The  internal  consistency  checks  are  quite  different  from  the  cross  validation  pro¬ 
cedure  described  in  Sect.  4.  In  particular,  each  observation  is  compared  with  the 
entire  set  of  observations  used  in  the  assimilation,  not  just  nearby  observations. 
A  metric  is  devised  to  test  whether  observation  innovations  are  likely  or  unlikely 
with  respect  to  other  observations  and  the  specified  background  and  observation 
error  statistics.  Once  the  decision  to  reject  an  observation  is  made  in  the  internal 
data  check  it  is  necessary  to  intervene  in  the  assimilation  process  to  ensure  that  the 
rejected  observation  has  no  effect  on  the  analysis.  Typically,  internal  data  checks 
are  performed  in  variational  analysis  schemes,  where  the  solution  is  obtained  us¬ 
ing  iterative  methods  that  can  be  interrupted  and  started  up  again.  The  internal  data 
checks  described  below  were  developed  for  the  Navy  Atmospheric  Variational  Data 
Assimilation  System  (NAVDAS),  described  in  Daley  and  Barker  (2001).  These 
checks  have  also  been  implemented  in  the  Navy  Coupled  Ocean  Data  Assimila¬ 
tion  (NCODA)  system  (Cummings  2005),  which  has  recently  been  updated  to  a 
3D  variational  analysis  based  on  NAVDAS.  The  discussion  below  is  adapted  from 
Daley  and  Barker  (2001,  Chap.  9.3). 

In  an  observation  based  analysis  system  the  analyzed  increments  (or  correction 
vector)  are  computed  according  to, 

(Xa  -  Xb )  =  BHt(HRHt  +  R)  l[y  -  H(x„)}  (4.8) 

where  xa  is  the  analysis  and  xb  is  the  forecast  model  background.  In  the  right  hand 
side  of  Fq.  (4.8),  B  is  the  background  error  covariance,  H  is  the  forward  operator, 
R  is  the  observation  error  covariance,  y  is  the  observation  vector,  and  T  indicates 
matrix  transpose.  The  observation  vector  contains  all  of  the  synoptic  temperature, 
salinity  and  velocity  observations  that  are  within  the  geographic  and  time  domains 
of  the  forecast  model  grid  and  update  cycle.  When  the  analysis  variable  and  the 
model  prognostic  variable  are  the  same  type,  the  forward  operator  H  is  simply  spa¬ 
tial  interpolation  of  the  forecast  model  grid  to  the  observation  location  performed 
in  three  dimensions.  Thus,  HBHT  is  approximated  directly  by  the  background  error 
correlation  between  observation  locations,  and  BHT  directly  by  the  error  correlation 
between  observation  and  grid  locations.  The  quantity  (y  —  H(xb)|  is  referred  to  as  the 
innovation  vector  (model-data  misfits  at  the  observation  locations). 

The  first  part  of  the  internal  data  check  uses  a  tolerance  lim  it.  Denote  A  =  H  BHT+  R 
as  the  observation  symmetric  positive  definite  matrix  of  Eq.  (4.8).  Define  A*=diag 
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(A).  Then,  define  the  observation  vector  (T  =  A~  ,/2[y-H(xb)].  The  elements  of  d* 
are  the  normalized  innovations  and  should  be  distributed  (over  many  realizations) 
in  a  normal  distribution  with  a  standard  deviation  equal  to  1 .0  if  the  background  and 
observation  error  covariances  have  been  specified  correctly.  Assuming  this  to  be  the 
case,  tolerance  limits  (TL)  are  defined.  Since  B  and  R  are  never  perfectly  known,  it 
is  best  to  use  a  relatively  high  tolerance  limit  (say,  TL=4.0)  in  operations.  The  test 
statistic  is  designed  to  identify  a  marginally  acceptable  observation  if  its  element  of 
d*  is  larger  than  the  specified  tolerance  limit. 

The  second  part  of  the  internal  data  check  is  a  consistency  check,  It  compares 
marginally  acceptable  observations  with  every  other  observation.  The  procedure  is 
a  logical  extension  of  the  tolerance  limit  check  described  above.  Define  the  vec¬ 
tor  d*  =  A',/2|y-/f(xb)|.  The  elements  of  d*  are  like  those  of  d*,  dimensionless 
quantities  normally  distributed.  However,  because  d*  involves  the  full  covariance 
matrix  A,  it  includes  correlations  between  all  of  the  observations.  By  comparing  the 
vectors  d*  and  d*  it  can  be  shown  which  marginally  acceptable  observations  are 
inconsistent  with  other  observations  and  can  therefore  be  rejected.  The  d*  metric 
should  increase  (decrease)  with  respect  to  d"  when  that  observation  is  inconsistent 
(consistent)  with  other  observations,  as  specified  by  the  background  and  observa¬ 
tion  error  statistics. 

The  internal  data  check  is  illustrated  using  the  example  given  in  Table  4.6  for  3 
hypothetical  observations  considered  marginally  acceptable  on  the  basis  of  a  pre¬ 
scribed  tolerance  limit  (d~)  check  value  of  3.0  (Daley  and  Barker  2001).  The  d* 
metric  for  the  first  observation  is  reduced  when  additional,  correlated  (/>  =  0.8)  ob¬ 
servations  more  accurate  than  the  background  (so  =  0  1)  are  considered.  In  this  case, 
the  suspect  observation,  rejected  individually  on  the  basis  of  the  tolerance  limit 
check,  is  now  determined  to  be  consistent  and  is  retained  in  the  analysis  (d*  =  1.9). 
However,  if  the  additional  data  are  uncorrelated  (/)  =  ~0.4)  while  also  being  accurate 
(£o=0.1),  then  the  results  indicate  the  suspect  observation  is  much  more  unlikely 
than  the  tolerance  limit  check  and  should  be  rejected  (d*  =  5.8).  Inaccurate  obser¬ 
vations  relative  to  the  background  (£q=2.0)  show  less  sensitivity  to  correlations 
among  observations  but  still  give  the  same  direction  of  change  (d*  vs.  d~)  as  the 
accurate  observations. 

There  are  difficulties  applying  the  consistency  data  check  in  practice  since  it 
requires  calculating  the  entire  A  1/2  matrix,  which  is  prohibitive  for  very'  large  prob¬ 
lems.  Fortunately,  there  are  some  good  approximations  to  this  calculation  that  can 
be  used  (Daley  and  Barker  2001).  However,  other  implementation  issues  remain.  To 


Table  4.6  Hypothetical  test 
case  for  internal  consistency 
check  (from  Daley  and 
Barker  2001) 


> 

II 

> 

II 

> 

II 

3.0 

K’l 

't) 

II 

1 

O 

p  =  0.8 

cf> 

ii 

p 

5.8 

1.9 

*o  =  2.0 

3.5 

2.4 

dA,  d *  defined  in  text 

p  correlation  between  observations 

e  observation  error  normalized  by  the  background  error 
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reject  an  observation  a  large  constant  is  added  to  the  appropriate  diagonal  element 
of  the  HBHT+R  matrix.  This  modifies  the  matrix  in  such  a  way  as  to  effectively 
prevent  the  rejected  observation  from  affecting  the  analysis.  However,  if  this  op¬ 
eration  is  done  during  the  descent  iteration  then  the  modified  matrix  is  no  longer 
consistent  with  the  other  vectors  that  have  been  evolving  as  part  of  the  conjugate 
gradient  solution.  The  descent  can  be  restarted  (very  expensive)  or  the  conjugate 
gradient  solution  vectors  can  be  suitably  altered  to  allow  the  descent  to  continue. 
In  either  case  the  tolerance  limit  and  internal  consistency  checks  can  be  applied 
multiple  times  during  the  descent  as  the  solution  resolves  more  and  more  of  the 
observation  innovations. 

As  discussed  in  Daley  and  Barker  (2001),  modifications  to  this  procedure  can  be 
made  for  extreme  events  when  the  specified  background  error  statistics  are  likely  to 
be  incorrect.  Typically,  error  statistics  in  the  assimilation  are  produced  by  averaging 
time  series  of  innovations  and  forecast  differences  and  reflect  average,  rather  than 
extreme,  conditions  over  the  model  domain.  When  changes  are  occurring  in  the 
ocean  (such  as  an  eddy  shedding  or  frontal  meander  event)  the  background  error 
statistics  are  likely  to  be  larger  than  normal.  In  this  case,  a  tolerance  limit  specified 
too  low  could  reject  good  (and  very  important)  data.  One  option  for  dealing  with 
this  is  make  one  pass  through  the  tolerance  limit  check  and  compute  the  mode  of  the 
d*  values  over  some  limited  subareas  of  the  analysis  domain.  The  mode  is  a  better 
statistic  here  because  it  is  less  susceptible  to  outliers  than  the  mean.  If  the  subarea 
mode  is  much  greater  than  one,  then  it  can  be  concluded  that  there  are  serious 
discrepancies  between  the  observations  and  the  background  in  that  area.  In  such  a 
case,  to  avoid  spuriously  rejecting  good  data,  the  subarea  tolerance  limit  should  be 
increased  beyond  the  prescribed  value. 


4.7  Adjoint  Sensitivities 

Adjoint-based  observation  sensitivity,  initially  developed  in  Numerical  Weather 
Prediction  as  an  observation-targeting  tool,  provides  a  feasible  (all  at  once)  ap¬ 
proach  to  estimating  observation  impact  for  a  large  variety  of  datasets  and  indi¬ 
vidual  observations.  Observation  impact  is  calculated  in  a  two-step  process  that 
involves  the  adjoint  of  the  forecast  model  and  the  adjoint  of  the  assimilation  system. 
First,  a  cost  function  (J)  is  defined  that  is  a  scalar  measure  of  some  aspect  of  the 
forecast  error.  The  forecast  model  adjoint  is  used  to  calculate  the  gradient  of  the 
cost  function  with  respect  to  the  forecast  initial  conditions  (9J/9xt).  The  second 
step  is  to  extend  the  initial  condition  sensitivity'  gradient  from  model  space  to  ob¬ 
servation  space  using  the  adjoint  of  the  assimilation  procedure  (9J/9y  =  KT9J/9xa), 
where  K  =  BHT|HBHT+R)  1  is  the  Kalman  gain  matrix  of  Eq.  4.8.  The  adjoint  of 
K  is  given  by  KT=[HBHT+RJ  'HB.  The  only  difference  between  the  forward  and 
adjoint  of  the  analysis  system  is  in  the  post-multiplication  of  going  from  the  solu¬ 
tion  in  observation  space  to  grid  space.  The  solver  (HBHT+R]  is  symmetric  or 
self-adjoint  and  operates  the  same  way  in  the  forward  and  adjoint  directions.  Given 


118 


J.  A.  Cummings 


an  analysis  sensitivity  vector,  observation  impact  is  obtained  as  a  scalar  product  of 
the  observed  model-data  differences  and  the  sensitivity  of  the  forecast  error  to  those 
differences.  Observations  will  have  the  largest  impact  on  reducing  forecast  error 
when  the  observation  influences  the  initial  conditions  in  a  dynamically  sensitive 
area.  It  is  not  necessary  for  the  observation  to  produce  a  large  change  (i.e.,  innova¬ 
tion)  to  the  initial  conditions  for  it  to  have  a  large  forecast  impact  (Baker  and  Daley 
2000;  Langland  and  Baker  2004). 

If  the  assimilation  of  an  observation  has  made  the  forecast  issued  from  the  ana¬ 
lyzed  state  more  accurate  than  a  forecast  valid  at  the  same  time  but  issued  from  a 
prior  state,  then  the  observation  is  considered  to  have  a  beneficial,  positive  impact. 
All  assimilated  observations  are  expected  to  have  beneficial  impacts  on  correcting 
the  initial  conditions  and  thereby  improving  the  forecast  issued  from  the  analysis. 
However,  if  consistent  non-beneficial  impacts  are  found  for  a  particular  data  type  or 
observing  system,  then  that  may  indicate  data  quality  control  issues,  such  as  subtle 
instrument  drift  or  calibration  problems  that  otherwise  are  difficult  to  assess  when 
considering  the  data  in  isolation.  Thus,  the  adjoint-based  data  impact  procedure  is 
an  effective  tool  to  provide  quantitative  diagnostics  of  ocean  data  quality.  The  use 
of  adjoint  sensitivities  in  ocean  data  assimilation  and  ocean  data  quality  control  is 
still  an  active  area  of  research  and  development. 


4.8  Summary  and  Conclusions 

Effective  ocean  data  quality  control  is  a  difficult  problem.  Observations  are  imper¬ 
fect  and  prone  to  error.  Data  with  errors  that  are  not  described  by  the  assimilation 
system  through  the  error  covariance  matrices  need  to  be  eliminated  prior  to  the 
analysis.  Effective  quality  control,  therefore,  requires  a  set  of  pre-established,  stan¬ 
dardized  test  procedures,  with  results  of  the  procedures  clearly  associated  with  the 
data  values.  Effectiveness  in  turn  depends  on  the  reliability  of  the  standard(s)  and 
on  the  choices  made  for  measuring  goodness  of  fit. 

The  need  for  observation  quality  control  depends  on  the  use  being  made  of  the 
observations.  Users  of  quality  controlled  data  sets  have  a  wide  range  of  views  on 
the  most  appropriate  standards  and  on  the  appropriate  “tightness  of  fit”  demanded 
by  the  quality  control  procedures  (too  tight  increases  the  chance  of  erroneously 
rejecting  anomalous  features;  too  loose  increases  the  chance  of  accepting  bad  data). 
Indicators  of  data  quality  must  be  useful  for  determining  if  the  quality  controlled 
observations  are  appropriate  for  a  particular  purpose.  In  this  paper,  observation 
quality  control  is  performed  as  a  prelude  to  assimilation  of  the  observations  in  an 
ocean  forecast  system.  Using  this  definition,  the  best  ocean  data  quality  scheme  is 
that  which  leads  to  the  best  ocean  forecast. 

It  is  surprisingly  difficult  to  demonstrate  consistent  impact  from  the  quality 
control  of  individual  observations  in  an  analysis/forecast  system.  Quality  control, 
however,  is  very  important  in  data  monitoring:  collection  of  statistics  on  the  perfor- 
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rnance  of  observing  systems;  detection  of  observing  systems  that  are  not  performing 
as  expected;  and  feedback  to  the  data  providers  so  that  deficiencies  are  corrected. 
y\n  integrated,  end-to-end  quality  control  system,  therefore,  must  ensure  that  results 
of  the  quality  control  procedures  are  recorded  for  independent  analysis  and  later 
use.  If  the  quality  control  is  carried  out  well,  then  it  can  reduce  the  duplication  of 
effort  among  the  users  of  ocean  data  value  added  is  not  lost  or  misinterpreted.  At 
a  minimum,  a  comprehensive  database  of  raw  and  processed  observed  values,  inde¬ 
pendent  estimates  of  the  same  quantities,  and  quality  control  outcomes  is  needed. 
The  database  would  be  used  to  look  for  “unexpected”  behavior  in  observing  sys¬ 
tems,  and  allow  users  and  operators  of  quality  control  systems  to  identify  systematic 
problems  in  order  to  get  errors  in  the  data  collection  or  data  transmission  corrected. 
At  present,  there  are  few  agreed-upon  standards  for  real-time  ocean  data  quality 
control  and  very  few  cases  where  the  procedures  and  results  from  the  oceanographic 
centers  have  been  compared.  As  the  GODAE  operational  oceanographic  commu¬ 
nity  continues  to  develop  a  range  of  complex  ocean  analysis  and  prediction  systems, 
it  is  important  that  procedures  be  developed  for  routinely  assessing  the  effective¬ 
ness  of  ocean  data  quality  control  and  for  routinely  exchanging  statistics  from  the 
quality  control  processes  at  the  operational  centers.  A  start  on  this  process  has  begun 
with  the  GODAE  QC  intercomparison  project  (Smith  2003;  Cummings  et  al.  2009), 
which  initially  is  focusing  on  profile  data  types. 

The  fully  automated  ocean  data  quality  control  procedures  described  in  this  pa¬ 
per  are  limited  to  observation  data  types  that  are  routinely  assimilated  in  ocean 
forecast  models.  New  ocean  observing  systems  continue  to  be  deployed  and  new 
failure  modes  of  existing  observing  systems  continue  to  be  identified.  Examples  of 
new  observing  systems  include  HF  coastal  radars  and  microwave  measurements 
of  sea  surface  salinity  from  space.  Examples  of  new  instrument  failure  modes  are 
the  pressure  and  salinity  sensor  issues  associated  with  the  long-term,  autonomous, 
deployments  of  the  Argo  profiling  floats.  New  observation  error  models  need  to  be 
developed  for  the  automated  quality  control  of  new  data  types,  and  existing  error 
models  need  to  be  updated  to  detect,  and  correct,  new  instrument  failure  modes. 
The  validity  of  existing  and  new  automated  quality  control  procedures  must  be  con¬ 
tinually  confirmed  by  formal  statistical  tests  and  by  examining  differences  between 
automated  and  delayed-mode  quality  control  outcomes  on  the  same  observation. 
The  automated  quality  control  system  can  be  considered  to  have  performed  well 
if  decisions  made  on  observations  in  real-time  are  consistent  with  decisions  made 
to  modify  or  reject  the  same  observations  in  delayed  mode,  where  more  rigorous 
scientific  and  expert  manual  intervention  quality  control  methods  are  possible.  De¬ 
layed  mode  quality  control  outcomes  of  the  Argo  profiling  float  array  are  readily 
available  and  can  be  used  in  this  evaluation.  This  activity  is  an  integral  component 
of  the  GODAE  QC  intercomparison  project,  which  includes  participation  from  the 
following  operational  centers:  Bureau  of  Meteorology  in  Australia,  Coriolis  Data 
Center  in  France,  the  Integrated  Science  Data  Management  Branch  in  Canada,  Fleet 
Numerical  Meteorology  and  Oceanography  Center  in  the  U.S.A,  and  the  Met  Office 
in  the  U.K. 
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